Method, apparatus and computer program product for joining the displays of multiple devices

ABSTRACT

A method is provided for enhancing a user interface with a mobile device by joining the displays of multiple devices together to function together with one another. In particular, example methods may provide for detecting a first touch or drag event, receiving a message indicating a second touch or drag event from a nearby device occurring during the first touch or drag event, creating a join event, and providing for transmission of the join event to the nearby device. The method may also include updating a registry of nearby devices in response to receiving a message regarding a touch or drag event from a nearby device. The method may further include providing for presentation of content on a display of a device and direction of content to be presented on a display of the nearby device.

FIELD OF INVENTION

Example embodiments of the present invention relate generally todisplays and user interfaces of mobile devices and, in particular, tousing multiple displays and/or user interfaces together to enhance thecapabilities of the mobile devices.

BACKGROUND

The modern communications era has brought about a tremendous expansionof wireline and wireless networks. Computer networks, televisionnetworks, and telephone networks are experiencing an unprecedentedtechnological expansion, fueled by consumer demand. Wireless and mobilenetworking technologies have addressed consumer demands while providingmore flexibility and immediacy of information transfer.

Mobile devices, such as cellular telephones, have become smaller andlighter while also becoming more capable of performing tasks that farexceed a traditional voice call. Mobile devices are increasinglybecoming small, portable computing devices that are capable of running avariety of applications and providing a user with a display on whichthey may watch video, view web pages, play interactive games, or readtext. Devices are often small enough to fit into a pocket to achievedesired portability of these devices; however, as the capabilities ofthe devices increases, the displays of such devices are used to displaylarge amounts of information and view objects which have traditionallybeen displayed on larger, less portable displays. It may be desirable toincrease the usable display size of a mobile device while maintainingthe portability.

BRIEF SUMMARY

In general, exemplary embodiments of the present invention provide animproved method of enhancing a user interface with a mobile device byjoining the displays of multiple devices together to function togetherwith one another. In particular, the method of example embodimentsprovides for detecting a first touch or drag even, receiving a messageindicating a second touch or drag event from a nearby device occurringduring the first touch or drag event, creating a join event, andproviding for transmission of the join event to the nearby device. Themethod may further include receiving a message indicating the end of thesecond touch or drag event from the nearby device. The method may alsoinclude updating a registry of nearby devices in response to receiving amessage regarding a touch or drag event from a nearby device. The joinevent may include join event rules and the join event rules may includedata regarding content to be displayed on the nearby device. Detectingthe first touch or drag event may include determining a drag eventdirection. The method may further include providing for presentation ofcontent on a display of a device and providing for direction of contentto be presented on a display of the nearby device, wherein the contentto be presented on the display of the nearby device is related to thecontent presented on the display of the device.

According to another embodiment of the present invention, an apparatusis provided. The apparatus may include at least one processor and atleast one memory including computer program code. The at least onememory and the computer program code may be configured to, with the atleast one processor, cause the apparatus to detect a first touch or dragevent, receive a message indicating a second touch or drag event from anearby device occurring during the first touch or drag event, create ajoin event, and provide for transmission of the join event to the nearbydevice. The apparatus may also be configured to receive a messageindicating the end of the second touch or drag event from the nearbydevice. The apparatus may also be configured to update a registry ofnearby devices in response to receiving a message regarding a touch ordrag event from a nearby device. The join event may include join eventrules and the join event rules may include data regarding content to bedisplayed on the nearby device. Detecting the first touch or drag eventmay include determining a drag event direction. The apparatus may alsobe configured to provide for presentation of content on a display of adevice and provide for direction of content to be presented on a displayof the nearby device, wherein the content to be presented on the displayof the nearby device is related to the content presented on the displayof the device. Detecting the first touch or drag event may includedetecting the direction of a touch moving from a first position to asecond position on a touch-sensitive display.

A further embodiment of the invention may include a computer programproduct including at least one computer-readable storage medium havingcomputer-executable program code instructions stored therein, thecomputer-executable program code instructions may include program codeinstructions for detecting a first touch or drag event, receiving amessage indicating a second touch or drag event from a nearby deviceoccurring during the first touch or drag event, creating a join event,and program code instructions for providing for transmission of the joinevent to the nearby device. The computer program product may furtherinclude program code instructions for receiving a message indicating theend of the second touch or drag event from the nearby device. Thecomputer program product may also include program code instructions forupdating a registry of nearby devices in response to receiving a messageregarding a touch or drag event from a nearby device. The join event mayinclude join event rules wherein the join event rules may include dataregarding content to be displayed on the nearby device. The program codeinstructions for detecting a touch or drag event may include programcode instructions for determining a drag event direction. The computerprogram product may further include program code instructions forpresentation of content on a display of a device and program codeinstructions for directing content to be presented on a display of thenearby device, wherein the content to be presented on the display of thenearby device is related to, the content presented on the display of thedevice. The program code instructions for detecting the first touch ordrag event may include program code instructions for detecting thedirection of a touch moving from a first position to a second positionon a touch-sensitive display.

BRIEF DESCRIPTION OF THE DRAWING(S)

Having thus described the invention in general terms, reference will nowbe made to the accompanying drawings, which are not necessarily drawn toscale, and wherein:

FIG. 1 illustrates an communication system in accordance with an exampleembodiment of the present invention;

FIG. 2 is a schematic block diagram of a mobile device according to anexample embodiment of the present invention;

FIG. 3 illustrates an embodiment of a method for joining the displays ofmultiple mobile devices according to example embodiments of the presentinvention;

FIG. 4 is a flow chart of a method of joining the displays of multiplemobile devices according to an example embodiment of the presentinvention;

FIG. 5 illustrates an embodiment of a method for joining the displays ofmultiple mobile devices according to another example embodiment of thepresent invention;

FIG. 6 illustrates an embodiment of the displays of multiple devicesjoined according to an example embodiment of the present invention;

FIG. 7 illustrates a mobile device configured for implementing exampleembodiments of the present invention;

FIG. 8 illustrates an embodiment of the displays of two mobile devicesjoined according to another example embodiment of the present invention;

FIG. 9 illustrates an embodiment of the displays of two mobile devicesjoined according to still another example embodiment of the presentinvention;

FIG. 10 illustrates an embodiment of a method for joining the displaysof three mobile devices according to an example embodiment of thepresent invention; and

FIG. 11 illustrates an embodiment of a method for joining the displaysof four mobile devices according to another example embodiment of thepresent invention.

DETAILED DESCRIPTION

Some example embodiments of the present invention will now be describedmore fully hereinafter with reference to the accompanying drawings, inwhich some, but not all embodiments of the invention are shown. Indeed,various embodiments of the invention may be embodied in many differentforms and should not be construed as limited to the example embodimentsset forth herein; rather, these example embodiments are provided so thatthis disclosure will satisfy applicable legal requirements. Likereference numerals refer to like elements throughout. As used herein,the terms “data,” “content,” “information” and similar terms may be usedinterchangeably to refer to data capable of being transmitted, receivedand/or stored in accordance with embodiments of the present invention.

Additionally, as used herein, the term ‘circuitry’ refers to (a)hardware-only circuit implementations (e.g., implementations in analogcircuitry and/or digital circuitry); (b) combinations of circuits andcomputer program product(s) comprising software and/or firmwareinstructions stored on one or more computer readable memories that worktogether to cause an apparatus to perform one or more functionsdescribed herein; and (c) circuits, such as, for example, amicroprocessor(s) or a portion of a microprocessor(s), that requiresoftware or firmware for operation even if the software or firmware isnot physically present. This definition of ‘circuitry’ applies to alluses of this term herein, including in any claims. As a further example,as used herein, the term ‘circuitry’ also includes an implementationcomprising one or more processors and/or portion(s) thereof andaccompanying software and/or firmware. As another example, the term‘circuitry’ as used herein also includes, for example, a basebandintegrated circuit or applications processor integrated circuit for amobile phone or a similar integrated circuit in a server, a cellularnetwork device, other network device, and/or other computing device.

A session may be supported by a network 30 as shown in FIG. 1 that mayinclude a collection of various different nodes, devices or functionsthat may be in communication with each other via corresponding wiredand/or wireless interfaces or in ad-hoc networks such as thosefunctioning over Bluetooth®. As such, FIG. 1 should be understood to bean example of a broad view of certain elements of a system that mayincorporate example embodiments of the present invention and not an allinclusive or detailed view of the system or the network 30. Although notnecessary, in some example embodiments, the network 30 may be capable ofsupporting communication in accordance with any one or more of a numberof first-generation (1G), second-generation (2.G), 2.5G,third-generation (3G), 3.5G, 3.9G, fourth-generation (4G) mobilecommunication protocols and/or the like.

One or more communication terminals such as the mobile terminal 10 andthe second mobile terminal 20 may be in communication with each othervia the network 30 and each may include an antenna or antennas fortransmitting signals to and for receiving signals from a base site,which could be, for example a base station that is part of one or morecellular or mobile networks or an access point that may be coupled to adata network, such as a local area network (LAN), a metropolitan areanetwork (MAN), and/or a wide area network (WAN), such as the Internet.In turn, other devices (e.g., personal computers, server computers orthe like) may be coupled to the mobile terminal 10 and the second mobileterminal 20 via the network 30. By directly or indirectly connecting themobile terminal 10 and the second mobile terminal 20 and other devicesto the network 30, the mobile terminal 10 and the second mobile terminal20 may be enabled to communicate with the other devices or each other,for example, according to numerous communication protocols includingHypertext Transfer Protocol (HTTP) and/or the like, to thereby carry outvarious communication or other functions of the mobile terminal 10 andthe second mobile terminal 20, respectively.

In example embodiments, either of the mobile terminals may be mobile orfixed communication devices. Thus, for example, the mobile terminal 10and the second mobile terminal 20 could be, or be substituted by, any ofpersonal computers (PCs), personal digital assistants (PDAs), wirelesstelephones, desktop computer, laptop computer, mobile computers,cameras, video recorders, audio/video players, positioning devices, gamedevices, television devices, radio devices, or various other devices orcombinations thereof.

Although the mobile terminal 10 may be configured in various manners,one example of a mobile terminal that could benefit from embodiments ofthe invention is depicted in the block diagram of FIG. 2. While severalembodiments of the mobile terminal may be illustrated and hereinafterdescribed for purposes of example, other types of mobile terminals, suchas portable digital assistants (PDAs), pagers, mobile televisions,gaming devices, all types of computers (e.g., laptops or mobilecomputers), cameras, audio/video players, radio, global positioningsystem (GPS) devices, or any combination of the aforementioned, andother types of communication devices, may employ embodiments of thepresent invention. As described, the mobile terminal may include variousmeans for performing one or more functions in accordance withembodiments of the present invention, including those more particularlyshown and described herein. It should be understood, however, that amobile terminal may include alternative means for performing one or morelike functions, without departing from the spirit and scope of thepresent invention.

The mobile terminal 10 illustrated in FIG. 2 may include an antenna 32(or multiple antennas) in operable communication with a transmitter 34and a receiver 36. The mobile terminal may further include an apparatus,such as a processor 40, that provides signals to and receives signalsfrom the transmitter and receiver, respectively. The signals may includesignaling information in accordance with the air interface standard ofthe applicable cellular system, and/or may also include datacorresponding to user speech, received data and/or user generated data.In this regard, the mobile terminal may be capable of operating with oneor more air interface standards, communication protocols, modulationtypes, and access types. By way of illustration, the mobile terminal maybe capable of operating in accordance with any of a number of first,second, third and/or fourth-generation communication protocols or thelike. For example, the mobile terminal may be capable of operating inaccordance with second-generation (2G) wireless communication protocolsIS-136, GSM and IS-95, or with third-generation (3G) wirelesscommunication protocols, such as UMTS, CDMA2000, wideband CDMA (WCDMA)and time division-synchronous CDMA (TD-SCDMA), with 3.9G wirelesscommunication protocols such as E-UTRAN (evolved-UMTS terrestrial radioaccess network), with fourth-generation (4G) wireless communicationprotocols or the like.

It is understood that the apparatus, such as the processor 40, mayinclude circuitry implementing, among others, audio and logic functionsof the mobile terminal 10. The processor may be embodied in a number ofdifferent ways. For example, the processor may be embodied as variousprocessing means such as processing circuitry, a coprocessor, acontroller or various other processing devices including integratedcircuits such as, for example, an ASIC (application specific integratedcircuit), an FPGA (field programmable gate array), a hardwareaccelerator, and/or the like. In an example embodiment, the processormay be configured to execute instructions stored in a memory device orotherwise accessible to the processor. As such, the processor may beconfigured to perform the processes, or at least portions thereof,discussed in more detail below with regard to FIG. 4. The processor mayalso include the functionality to convolutionally encode and interleavemessage and data prior to modulation and transmission. The processor mayadditionally include an internal voice coder, and may include aninternal data modem.

The mobile terminal 10 may also comprise a user interface including anoutput device such as an earphone or speaker 44, a ringer 42, amicrophone 46, a display 48, and a user input interface, which may becoupled to the processor 40. The user input interface, which allows themobile terminal to receive data, may include any of a number of devicesallowing the mobile terminal to receive data, such as a keypad 50, atouch sensitive display (not shown) or other input device. Inembodiments including the keypad, the keypad may include numeric (0-9)and related keys (#, *), and other hard and soft keys used for operatingthe mobile terminal 10. Alternatively, the keypad may include aconventional QWERTY keypad arrangement. The keypad may also includevarious soft keys with associated functions. In addition, oralternatively, the mobile terminal may include an interface device suchas a joystick or other user input interface. The mobile terminal mayfurther include a battery 54, such as a vibrating battery pack, forpowering various circuits that are used to operate the mobile terminal,as well as optionally providing mechanical vibration as a detectableoutput.

The mobile terminal 10 may further include a user identity module (UIM)58, which may generically be referred to as a smart card. The UIM may bea memory device having a processor built in. The UIM may include, forexample, a subscriber identity module (SIM), a universal integratedcircuit card (UICC), a universal subscriber identity module (USIM), aremovable user identity module (R-UIM), or any other smart card. The UIMmay store information elements related to a mobile subscriber. Inaddition to the UIM, the mobile terminal may be equipped with memory.For example, the mobile terminal may include volatile memory 60, such asvolatile Random Access Memory (RAM) including a cache area for thetemporary storage of data. The mobile terminal may also include othernon-volatile memory 62, which may be embedded and/or may be removable.The non-volatile memory may additionally or alternatively comprise anelectrically erasable programmable read only memory (EEPROM), flashmemory or the like. The memories may store any of a number of pieces ofinformation, and data, used by the mobile terminal to implement thefunctions of the mobile terminal. For example, the memories may includean identifier, such as an international mobile equipment identification(IMEI) code, capable of uniquely identifying the mobile terminal.Furthermore, the memories may store instructions for determining cell idinformation. Specifically, the memories may store an application programfor execution by the processor 40, which determines an identity of thecurrent cell, i.e., cell id identity or cell id information, with whichthe mobile terminal is in communication.

An example embodiment of a communication network in accordance with oneexample embodiment is presented by FIG. 1. A flowchart illustratingoperations performed by or in relation to the network of an exampleembodiment is presented in FIG. 4 and may be performed, for example, bythe mobile terminal such as shown in FIG. 2. It will be understood thateach block of the flowcharts, and combinations of blocks in theflowcharts, may be implemented by various means, such as hardware,firmware, processor, circuitry and/or other device(s) associated withexecution of software including one or more computer programinstructions. For example, one or more of the procedures described abovemay be embodied by computer program instructions. In this regard, thecomputer program instructions which embody the procedures describedabove may be stored by a memory device 60, 62 of an apparatus, such as aclient, employing an example embodiment of the present invention andexecuted by a processor 40 in the apparatus. As will be appreciated, anysuch computer program instructions may be loaded onto a computer orother programmable apparatus (e.g., hardware), such as depicted in FIG.2, to produce a machine, such that the resulting computer or otherprogrammable apparatus embody means for implementing the functionsspecified in the flowchart block(s). These computer program instructionsmay also be stored in a computer-readable memory that may direct acomputer or other programmable apparatus, e.g., the overlay networkhost, to function in a particular manner, such that the instructionsstored in the computer-readable memory produce an article of manufacturethe execution of which implements the function specified in theflowchart block(s). The computer program instructions may also be loadedonto a computer or other programmable apparatus to cause a series ofoperations to be performed on the computer or other programmableapparatus to produce a computer-implemented process such that theinstructions which execute on the computer or other programmableapparatus implement the functions specified in the flowchart block(s).

In general, example embodiments of the present invention provide amethod for enhancing and/or expanding a display and/or user interface ofa mobile terminal using neighboring mobile terminals. For example, afirst mobile terminal may display a web page, an application, or otherobject on a touch-sensitive display and upon detection of a particularmotion recognized by the first mobile terminal and a nearby secondmobile terminal, the displays may be virtually linked to display theobject from the first mobile terminal.

Example embodiments of the present invention are described herein withreference to a mobile terminal comprising a touch-sensitive display(e.g., a touchscreen); however, embodiments of the present invention maybe configured to be operable on various types of mobile terminals withsingle or multi-touch displays, displays with separate touch-paduser-interfaces, or other display types.

Embodiments of the present invention may comprise at least twofundamental operations. A first operation includes detecting an actionthat signals that the mobile terminal, such as mobile terminal 10,should be joined with a second mobile terminal. The action may include apinch gesture while the fingers performing the pinch gesture are incontact with the touch-sensitive displays of the two mobile terminals.The second operation may allow the user interfaces of both of the mobileterminals to be modified according to a set of join event rules that maydictate how the joining of the user interfaces is executed based uponthe first operation and the orientation of the mobile terminals. Thejoin event rules may be dictated by the application(s) in use in one orboth of the mobile terminals.

An example embodiment of the present invention is illustrated in FIG. 3in which a first mobile terminal 300 and a second mobile terminal 310are placed adjacent, or in close proximity to one another. A user 320may use a first digit 322, such as a thumb, and a second digit 324, suchas an index finger in a pinching motion (indicated by arrow 330) whilein contact with a respective touch-sensitive display 305, 315. While theillustrated embodiment depicts a user's 320 fingers 322, 324, it shouldbe appreciated that any touching device may be used, such as a stylus,on the touch-sensitive displays 305, 315. Further, the joining actionmay also be performed by fingers of two different hands (of the same orseparate users), or touching devices belonging to two different users.The motion of the fingers 322, 324 may cause the terminals to join thetouch-sensitive displays in a manner dictated by their orientation andthe way in which the touch-sensitive displays were touched.

An example embodiment of a method of the present invention in whichmultiple user interfaces are joined together to operate in cooperationis shown in the flow chart of FIG. 4 where two terminals, mobileterminal A and mobile terminal B, are in close proximity to one another,and perhaps in physical contact. Mobile terminal A, may detect a touchevent at coordinate location (x1, y1) of the touch-sensitive display (ortouchscreen) of mobile terminal A at 401. Mobile terminal A may thentransmit a network message to nearby devices (e.g., other mobileterminals) through a near-field communications method (such asBluetooth® or Zigbee®). At substantially the same time (e.g., within arelatively small amount of time, such as 0.25 seconds), mobile terminalB may detect a touch event at coordinate location (s1, t1) of thetouch-sensitive display of mobile terminal B at 402. Mobile terminal Bmay receive the message from mobile terminal A transmitted at 401indicating that there is an ongoing touch event from (x1, y1) and mobileterminal B may update its registry of nearby devices accordingly at 403.Mobile terminal A may receive the message from mobile terminal B,transmitted at 402, indicating that there is an ongoing touch event from(s1, t1) and mobile terminal A may update its registry of nearby devicesaccordingly at 404. Mobile terminal A may detect that the touch event,detected at 401, is actually a drag event where the touch point hasmoved from (x1, y1) to (x2, y2) at 405. Mobile terminal A may determinewhich edge of the screen the drag event is moving toward: top, left,right, or bottom. For example, if x2−x1>|y2−y1|, it may be deduced thatthe drag event is toward the right edge of the screen where thecoordinate system has an origin at the lower left corner of the screen,the positive x-axis is toward the right side of the screen, and thepositive y-axis is toward the top of the screen. A network event aboutthis may be sent to other nearby devices. Mobile terminal B may receivethe message about the drag event on mobile terminal A toward the rightedge and mobile terminal B may update its registry accordingly at 406.Mobile terminal B may detect that its touch event has become a dragevent where the touch point has moved from (s1, t1) to (s2, t2) at 407.For purposes of the example herein, the direction may be calculated tobe toward the bottom of the screen. Mobile terminal A may receive themessage about the drag event on mobile terminal B and update theregistry of the states of nearby devices accordingly at 408. Mobileterminal B may recognize that the drag event has ended as the user liftstheir finger off of the display and mobile terminal B may send a networkevent about the end of the drag event to nearby devices at 409. Mobileterminal B may then check the registry of the states of nearby devicesto determine if there are other simultaneous drag events occurring onany nearby devices. Mobile terminal B may recognize that mobile terminalA experienced a simultaneous drag event and the direction of the dragevent of mobile terminal A. A join event may be created by mobileterminal B and transmitted to mobile terminal A at 411. Mobile terminalA may receive a message from mobile terminal B that the drag event ofMobile terminal B has ended and mobile terminal A may update itsregistry accordingly. Mobile terminal A may further receive the joinevent and rules of the join event from mobile terminal B at 412. Mobileterminal A may then apply the join event to its touch-sensitive displaysuch that the touch-sensitive displays of mobile terminal A and mobileterminal B appear joined or cooperative based on the rules associatedwith the join event at 413. Mobile terminal A may recognize that thedrag event on the touch-sensitive display of mobile terminal A has endedand transmits a signal indicating this event to nearby devices. Mobileterminal B receives the message from mobile terminal A and updates theregistry of mobile terminal B accordingly.

In the aforementioned embodiment, the join event may be initiated bywhichever device detects the lifting of the finger earlier, which thenbecomes the dominant device. In the embodiment depicted in FIG. 4, thefinger or pointing device was lifted from mobile terminal B first suchthat mobile terminal B initiated the join event and becomes the dominantdevice. Mobile terminal A becomes the secondary device presenting theobject of the dominant device according to the rules of the join eventreceived from the dominant device. In the event that both devices detectthe removal of the finger or end of their respective drag eventssubstantially simultaneously, one of the join events will be detected asredundant. In the case of redundant join events, a request for userinput to identify the dominant device may be presented on one or bothdevices. Optionally, the dominant device may be dictated by theapplication in use on the devices involved in the join event such thatthe join gesture has no impact on the determination of the dominantdevice.

FIG. 4 is a flowchart of systems, methods and program products accordingto example embodiments of the invention. It will be understood that eachblock of the flowchart, and combinations of blocks in the flowchart, maybe implemented by various means, such as hardware, firmware, processor,circuitry and/or other device associated with execution of softwareincluding one or more computer program instructions. For example, one ormore of the procedures described above may be embodied by computerprogram instructions. In this regard, the computer program instructionswhich embody the procedures described above may be stored by a memorydevice of an apparatus employing an embodiment of the present inventionand executed by a processor in the apparatus. As will be appreciated,any such computer program instructions may be loaded onto a computer orother programmable apparatus (e.g., hardware) to produce a machine, suchthat the resulting computer or other programmable apparatus embody meansfor implementing the functions specified in the flowchart block(s).These computer program instructions may also be stored in acomputer-readable memory that may direct a computer or otherprogrammable apparatus to function in a particular manner, such that theinstructions stored in the computer-readable memory produce an articleof manufacture the execution of which implements the function specifiedin the flowchart block(s). The computer program instructions may also beloaded onto a computer or other programmable apparatus to cause a seriesof operations to be performed on the computer or other programmableapparatus to produce a computer-implemented process such that theinstructions which execute on the computer or other programmableapparatus provide operations for implementing the functions specified inthe flowchart block(s).

Accordingly, blocks of the flowchart support combinations of means forperforming the specified functions, combinations of operations forperforming the specified functions and program instruction means forperforming the specified functions. It will also be understood that oneor more blocks of the flowchart, and combinations of blocks in theflowcharts, can be implemented by special purpose hardware-basedcomputer systems which perform the specified functions, or combinationsof special purpose hardware and computer instructions.

In an example embodiment, an apparatus for performing the methods ofFIG. 4 above may comprise a processor (e.g., the processor 70)configured to perform some or each of the operations (401-413) describedabove. The processor may, for example, be configured to perform theoperations (401-413) by performing hardware implemented logicalfunctions, executing stored instructions, or executing algorithms forperforming each of the operations. Alternatively, the apparatus maycomprise means for performing each of the operations described above. Inthis regard, according to an example embodiment, examples of means forperforming operations 401-413 may comprise, for example, the processor70, the resource manager 80, and/or a device or circuit for executinginstructions or executing an algorithm for processing information asdescribed above.

The nature of the join event, and the rules associated thereto may beapplication dependent in order to optimize the application or objectthat is displayed on the dominant device. If a pinch gesture is used toinitiate a join event, the devices involved in the join event may beaware of the drag events and toward which side of a touch-sensitivedisplay they each occurred. If the drag event of the dominant device isto the right side of the touch-sensitive display while the drag event ofthe secondary device is to the bottom of the touch-sensitive display,the joining of the displays may include a virtual interface between theright edge of the touch-sensitive display of the dominant device and thebottom edge of the touch-sensitive display of the secondary device. Theparticular location of the drag events on the touch-sensitive displaysmay factor into some join events as illustrated in FIG. 5, wherein adrag event 512 on a dominant device 510 to the right edge of the topportion of a touch-sensitive display 515 and the drag event 522 to leftedge on the bottom portion of a touch-sensitive display 525 of asecondary device 520 may cause a staggered virtual interface between thetouch-sensitive displays. While join event rules of some applicationsmay only use the relevant side of the touch-sensitive display related tothe drag event and ignore the exact location, other applications may usethe exact locations of the drag events to determine alignment of thedevices to be joined.

As noted above, the nature of the join event and rules associatedthereto may be application dependent such that under certaincircumstances, mobile terminals that are joined may not be in adominant-secondary relationship, but rather a collaborativerelationship. Applications using a collaborative relationship mayinclude games or applications that require a symbiotic data orinformation exchange.

An example application for which embodiments of the present inventionmay be implemented includes a virtual mind map as displayed on a firstmobile terminal placed, for example, on a table top surface. A secondmobile terminal may be placed adjacent to the first mobile terminal anda user may perform a pinch gesture between the touch-sensitive displaysof each of the mobile terminals. The user may indicate through the pinchgesture or through a prompt by either terminal that the first mobileterminal is the dominant mobile terminal. The second terminal may thendisplay a portion of the virtual mind map that was previously off-screenof the first mobile terminal as the second mobile terminal may functionto expand the display area of the first mobile terminal. As noted above,the application, rather than the joining gesture, may be indicative ofthe dominant device. In the example application of a virtual mind map,the terminal displaying an assigned content-space viewing rectangle mayautomatically be recognized as the dominant device when joined with aterminal showing an overview of the virtual mind map and not yet havinga content-space viewing rectangle assigned.

A technical implementation of the joining event of the aforementionedexample application may include wherein the first mobile terminal isdisplaying a specific region of a larger virtual space, defined as arectangle in content-space coordinates. The second mobile terminal mayhave no such defined region and may be displaying an overview of allcontent. When a drag event or pinch gesture is detected between thefirst mobile terminal and the second mobile terminal, with the firstmobile terminal determined to be the dominant device, the join event issent to the second mobile terminal. As noted above, since the secondmobile terminal had no defined region, the application may cause thesecond mobile terminal to become the secondary device regardless of thenature of the join gesture. The join event rules may include thecoordinates of where the drag or pinch event occurred on the firstmobile terminal and the content-space rectangle being displayed on thefirst mobile terminal. The second mobile terminal may then display acontent-space view rectangle that is aligned with the drag or pinchevent that occurred between the two mobile terminals and supplements thedisplayed content-space view rectangle of the first mobile terminal.

Example embodiments of the join event may include join event rules orparameters to further define the join event and indicate how the mobileterminals involved in the join event are to interact. An exampleembodiment of a join event rule set may include relative orientationsand the contacted edges. While applications such as a map-viewer may beconducive to embodiments that expand the viewable area when mobileterminals are joined, applications, including map-viewers, may offeralternative uses for the secondary device. For example, a dominantdevice may be executing an application for display of a map of alocation. If the dominant device is joined with a secondary device bydragging to the right side of the dominant device touch-sensitivedisplay (e.g., the secondary device is on the right side of the dominantdevice), the secondary device may show an expanded view of the map;however, if the dominant device is joined with a secondary device bydragging to the left side of the dominant device touch-sensitive display(e.g., the secondary device is on the left side of the dominant device),the secondary device may display map options such as zoom functions,location search options, topographical information, populationstatistics, etc. The function performed by the secondary device inresponse to the join event may be dictated by how the drag or pinchevent is performed and the function that the secondary device is toperform may be included in the join event rules.

Another example embodiment of join event rules may include join eventrules that pertain to the orientation of the devices that are joined.For example, if a first mobile terminal includes a rectangulartouch-sensitive display that is oriented vertically and a second mobileterminal includes a rectangular touch-sensitive display that is orientedhorizontally, the relative orientation may affect how the join eventrules are created and/or implemented. For example, as illustrated inFIG. 6, an application that is displaying a web page on the display 615a first mobile terminal 610 may be joined with a secondary mobileterminal 620 that is oriented perpendicularly to the first mobileterminal. The second mobile terminal may be of an orientation that isbetter suited to web-page display such that the web page is displayed onthe display 625 of the second mobile terminal 620 while an overview ofthe entire webpage, together with an indication of what area is beingdisplayed on the second mobile terminal 620 is displayed on the firstmobile terminal 610, such as the viewing rectangle 630 of theillustrated embodiment. Scrolling on the display of the secondary mobileterminal 620 may be accomplished by pressing/touching the “more” icon627 of the second mobile terminal 620, or alternatively, by dragging theviewing rectangle 630 to the desired viewing area of the whole web pagedisplayed on the touch sensitive display 615 of the first mobileterminal 610. For example, a user may touch the “more” icon 627 and theywill be presented with the section of the web page displayed on thefirst mobile terminal 610 just below the viewing rectangle 630, and theviewing rectangle 630 will move accordingly. Optionally, a drag event ofdragging a finger or other touching device towards the bottom of thetouch sensitive display 625 of the second mobile terminal 620 may alsoscroll down the web page.

Further example embodiments of the present invention are illustrated inFIGS. 7-9, wherein an application, such as a Twitter® account, ispresented on the display 715 of mobile terminal 710. The mobile terminal810 (the dominant device in the depicted example) may be joined with asecondary device, mobile terminal 820, as illustrated in FIG. 8, withthe orientations of each device being parallel, joining the dominantdevice 810 through the bottom of the touch-sensitive display 815 and thesecondary device through the top of the touch-sensitive display 825. Theorientation of the mobile terminals 810, 820 and the application activeon the dominant device 810 may dictate the join event rules that aresent to the secondary device 820. In the illustrated embodiment, thesecondary device expands the display of the dominant device 810 todisplay additional information, and in the instant example, additionalmessages that were previously off-screen of the dominant device 810.FIG. 9 illustrates a mobile terminal 910 as the dominant device, similarto that of FIGS. 7 and 8; however, the secondary device (mobile terminal920) is oriented perpendicularly to the dominant device 910. When thedominant device 910 is joined through the bottom of the touch-sensitivedisplay 915 to the left side of the touch-sensitive display 925 of thesecondary device 920 that is oriented with its left side facing thedominant device 910, the join event rules sent to the secondary device920 may be different. In the illustrated embodiment, the join eventrules cause the secondary device to display options rather than acontinuation of the information that is displayed on the mobile terminal910. Further, the join event rules may be configured by a user such thatthe user determines how a join event is to occur based on the variablesof the drag direction, the application, and the relative orientations ofthe devices involved.

Example embodiments of the present invention may join any number ofmobile terminals as illustrated in FIGS. 10 and 11. FIG. 10 depictsthree mobile terminals situated side-by-side and joined through a pinchgesture, with the arrows each indicating a touch and touch direction.The order in which the mobile terminals are joined may dictate whichmobile terminal is the dominant device. For example, if mobile terminal1010 is joined with mobile terminal 1020, and mobile terminal 1010 isthe dominant device, when mobile terminal 1030 is joined to mobileterminal 1020, mobile terminal 1010 remains the dominant device and maydictate the join event rules based upon the application, the deviceorientation, and any previously joined devices, such as 1020. In such anembodiment, as mobile terminal 1020 detects the drag event, the dragevent is communicated to both 1010 and 1030 for updating theirregistries. Upon determination that a join event has been initiated,since mobile terminal 1020 is already a secondary device to mobileterminal 1010, mobile terminal 1010 may transmit the join event rules.Optionally, the join event rules that are transmitted to mobile terminal1020 when the join event is initiated between mobile terminal 1010 andmobile terminal 1020 may include join event rules for subsequentlyjoined mobile terminals such that mobile terminal 1020 may transmit thejoin event rules to mobile terminal 1030. FIG. 11 illustrates anotherexample embodiment wherein multiple devices are joined through more thanone join event.

As described above and as will be appreciated by one skilled in the art,embodiments of the present invention may be configured as a system,method or electronic device. Accordingly, embodiments of the presentinvention may be comprised of various means including entirely ofhardware or any combination of software and hardware. Furthermore,embodiments of the present invention may take the form of a computerprogram product on a computer-readable storage medium havingcomputer-readable program instructions (e.g., computer software)embodied in the storage medium. Any suitable computer-readable storagemedium may be utilized including hard disks, CD-ROMs, optical storagedevices, or magnetic storage devices.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the inventions are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

That which is claimed:
 1. A method comprising: detecting a first dragevent, in a first direction, which occurs on a display of a device;determining, from the detected first drag event and directioninformation included in a message transmitted wirelessly to the deviceby a nearby device, that a second drag event has occurred on a displayof the nearby device in a second direction that is substantiallyopposite to the first direction, wherein the first drag event and thesecond drag event correspond with a user performing a pinching motion;determining that the second drag event occurred during the first dragevent; creating a join event in response to determining at least thatthe second drag event occurred on the display of the nearby device inthe second direction substantially opposite to the first direction andthat the second drag event occurred during the first drag event; andproviding for transmission of the join event to the nearby device. 2.The method according to claim 1, wherein the join event includes joinevent rules and wherein the join event rules include data regardingcontent to be displayed on the nearby device.
 3. The method according toclaim 1, further comprising: providing for presentation of content onthe display of the device; and providing for direction of content to bepresented on the display of the nearby device, wherein the content to bepresented on the display of the nearby device is related to the contentpresented on the display of the device.
 4. The method according to claim1, wherein detecting the first drag event in the first directioncomprises detecting that the first drag event was directed towards afirst edge of the display of the device, and determining that the seconddrag event occurred in the second direction substantially opposite thefirst direction comprises determining that the second drag event wasdirected towards a second edge of the display of the nearby device. 5.The method according to claim 1, wherein join event rules for the joinevent define an interaction between the device and the nearby devicethat depends upon the relative orientation of the device and the nearbydevice.
 6. The method according to claim 5, wherein the device and thenearby device are joined according to first join event rules when thedevice and the nearby device have a first relative orientation and thedevice and the nearby device are joined according to a second join eventrules when the device and the nearby device have a second relativeorientation.
 7. The method according to claim 6, wherein in the firstrelative orientation, the device and the nearby device have the sameorientation, and in the second relative orientation, the nearby deviceis orientated perpendicularly to the device.
 8. An apparatus comprisingat least one processor and at least one memory including computerprogram code, the at least one memory and the computer program codeconfigured to, with the at least one processor, cause the apparatus toat least perform: detecting a first drag event, in a first direction,that occurs on a display of a device; determining, from the detectedfirst drag event and direction information included in a messagetransmitted wirelessly to the device by a nearby device, that a seconddrag event has occurred on a display of the nearby device in a directionthat is substantially opposite to the first direction, wherein the firstdrag event and the second drag event correspond with a user performing apinching motion; determining that the second drag event occurred duringthe first drag event; creating a join event in response to determiningat least that the second drag event occurred on the display of thenearby device in the second direction substantially opposite to thefirst direction and that the second drag event occurred during the firstdrag event; and providing for transmission of the join event to thenearby device.
 9. The apparatus according to claim 8, wherein the joinevent includes join event rules and wherein the join event rules includedata regarding content to be displayed on the nearby device.
 10. Theapparatus according to claim 8, wherein the at least one memory and thecomputer program code are configured to, with the at least oneprocessor, cause the apparatus to further perform: providing forpresentation of content on the display of the device; and providing fordirection of content to be presented on the display of the nearbydevice, wherein the content to be presented on the display of the nearbydevice is related to the content presented on the display of the device.11. The apparatus according to claim 8, wherein the type of informationprovided for display on the joined device is determined based upon thedirection of the second drag event.
 12. The apparatus according to claim8, wherein the at least one memory and the computer program code areconfigured to, with the at least one processor, cause the apparatus toexpand information presented on the display of the device during thefirst drag event in response to the join event.
 13. The apparatusaccording to claim 8, wherein detecting the first drag event in thefirst direction comprises detecting that the first drag event wasdirected towards a first edge of the display of the device, anddetermining that the second drag event occurred in the second directionsubstantially opposite the first direction comprises determining thatthe second drag event was directed towards a second edge of the displayof the nearby device.
 14. The apparatus according to claim 8, whereinjoin event rules for the join event define an interaction between thedevice and the nearby device that depends upon the relative orientationof the device and the nearby device.
 15. The apparatus according toclaim 14, wherein the device and the nearby device are joined accordingto first join event rules when the device and the nearby device have afirst relative orientation and the device and the nearby device arejoined according to a second join event rules when the device and thenearby device have a second relative orientation.
 16. The apparatusaccording to claim 15, wherein in the first relative orientation, thedevice and the nearby device have the same orientation, and in thesecond relative orientation, the nearby device is orientatedperpendicularly to the device.
 17. The apparatus according to claim 8,wherein the apparatus is the device.
 18. A computer program productcomprising at least one non-transitory computer-readable storage mediumhaving computer-executable program code instructions stored therein, thecomputer-executable program code instructions comprising: program codeinstructions for detecting a first drag event, in a first direction,which occurs on a display of a device; program code instructions fordetermining, from the detected first drag event and directioninformation included in a message transmitted wirelessly to the deviceby a nearby device, that a second drag event has occurred on a displayof the nearby device in a direction that is substantially opposite tothe first direction, wherein the first drag event and the second dragevent correspond with a user performing a pinching motion; program codefor determining that the second drag event occurred during the firstdrag event; program code instructions for creating a join event inresponse to determining at least that the second drag event occurred onthe display of the nearby device in the second direction substantiallyopposite to the first direction and that the second drag event occurredduring the first drag event; program code instructions for providing fortransmission of the join event to the nearby device.
 19. The computerprogram product according to claim 18, wherein the join event includesjoin event rules and wherein the join event rules include data regardingcontent to be displayed on the nearby device.
 20. The computer programproduct according to claim 18, further comprising program codeinstructions for providing for presentation of content on the display ofthe device and program code instructions for directing of content to bepresented on the display of the nearby device, wherein the content to bepresented on the display of the nearby device is related to the contentpresented on the display of the device.
 21. The computer program productaccording to claim 18, wherein detecting the first drag event in thefirst direction comprises detecting the first drag event was directedtowards a first edge of the display of the device, and determining thatthe second drag event occurred in the second direction, substantiallyopposite the first direction, comprises determining that the second dragevent was directed towards a second edge of the display of the nearbydevice.